Hypothesis testing and information theory

The testing of binary hypotheses is developed from an information-theoretic point of view, and the asymptotic per formance of opt imum hypothesis testers is developed in exact analogy to the asymptotic per formance of opt imum channel codes. The discrimination, introduced by Kullback, is developed in a role analogous to that of mutual information iu channel coding theory. Based on the discrimination, an error-exponent function e(r) is defined. This function is found to descr ibe the behavior of opt imum hypothesis testers asymptotically with block length. Next, mutual information is introduced as a minimum of a set of discriminations. This approach has later coding significance. The channel reliability-rate function E(R) is def ined in terms of discrimination, and a number of its mathematical propert ies developed. Sphere-packing-l ike bounds are developed in a relatively straightforward and intuitive manner by relating e(r) and E(R). This ties together the aforement ioned developments and gives a lower bound in terms of a hypothesis testing model. The result is valid for discrete or cont inuous probability distributions. The discrimination function is also used to define a source code reliability-rate function. This function allows a simpler proof of the source coding theorem and also bounds the code performance as a function of block length, thereby providing the source coding analog of E(R).